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1.0  Introduction 


This  report  summarizes  the  accomplishments  of  Frontier  Technology,  Inc.  (FTI)  under 
Contract  Number  F30602-99-C-0146.  The  primary  objectives  for  FTI  under  this  contract  were 
to  enhance  and  demonstrate  FTI’s  Air  Operations  Center  (AOC)  Model  in  and  to  integrate  a 
Space  Based  Radar  capability  with  the  distributed  Global  Awareness  Virtual  Test  Bed 
(GAVTB)  simulation  environment.  A  formal  Design  of  Experiment  (DOE)  process  was 
employed  to  develop  metrics  and  define  an  experiment  based  on  a  Time  Critical  Targeting 
(TCT)  scenario. 

The  major  enhancements  to  the  AOC  Model  included  development  of  an  HLA  interface, 
development  and  integration  of  the  embedded  Decision  Integrated  Support  Environment 
(DISE)  module,  and  integration  of  the  OPUS  route  planning  software.  AOC  Model 
development  has  followed  a  spiral  approach.  Each  spiral  has  added  new  capabilities  or 
improved  on  existing  capabilities.  The  initial  capability  was  developed  for  the  Collaborative 
Enterprise  Environment  (CEE)  under  subcontract  to  SAIC.  It  should  be  noted  that  some  of  the 
development  described  in  this  report  was  funded  by  contracts  other  than  GAVTB.  These 
include  SERENE,  SimBA,  and  ongoing  CEE  support.  All  four  efforts  have  been  able  to 
leverage  the  investments  of  the  other  programs,  to  the  benefit  of  each. 


2.0  AOC  Overview 

FTI’s  AOC  Model  is  a  quick-turn,  medium  fidelity  air  operations  command  and  control  (C2) 
simulation,  including  an  embedded  decision  support  module  for  target  prosecution.  AOC 
models  aircraft,  targets,  weapons,  routes,  and  Air  Tasking  Orders  (ATO’s).  It  runs  as  a  real¬ 
time  or  constructive  simulation.  AOC  participates  in  DoD  standard  High  Level  Architecture 
(HLA)  federations,  to  receive  aircraft  and  targets,  and  sends  ATO’s  and  revised  routes.  AOC 
also  participates  (concurrently  or  separately)  in  non-HLA  simulation  confederations  receiving 
target  detections  via  FTI’s  Automatic  Target  Recognition  (ATR)  Framework  and  Satellite  Tool 
Kit  (STK)  from  Analytical  Graphics,  Inc.  (AGI).  AOC’s  embedded  Bayesian-network  based 
decision  support  module  DISE  decides  if  time-sensitive  targets  are  to  be  prosecuted  and 
selects  the  best  assets  to  assign  for  attack.  AOC  integrates  the  OPUS  mission  planner  from 
ORCA,  for  generating  initial  and  retasked  aircraft  routes.  AOC  outputs  ATO’s  and  revised 
routes  via  HLA,  text  files,  or  network  socket  connections.  Figure  2-1  depicts  the  user 
interface. 

2.1  AOC  Features 

Command  and  Control  Simulation 

AOC  provides  variable  fidelity  modeling  and  simulation  of  processes  and  objects 
associated  with  command  and  control  (C2)  tasks: 

•  Aircraft 

•  Targets 

•  Weapons 

•  Air  Tasking  Orders  (ATOs) 

•  Routes 
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C2  timelines 


Figure  2-1  -  Air  Operations  Center  (AOC)  User  Interface 


Decision  Support 

Command  decision  support  is  provided  in  AOC  with  its  embedded  Decision  Integrated 
Support  Engine,  or  DISE,  which  is  applied  to  time-sensitive  target  prosecution  decisions. 
DISE  takes  into  account  many  decision  factors,  each  weighted  according  to  user  preferences, 
to  decide  if  a  new  time-sensitive  target  is  to  be  prosecuted,  and  which  available  aircraft  should 
be  sent  to  attack  it. 

Mission  Route  Planning 


Mission  route  planning  is  available  to  AOC  through  OPUS.  AOC  offers  the  flexibility  of 
either  importing  OPUS  routes  generated  a  priori,  or  dynamically  interfacing  to  OPUS  to  create 
routes  at  run-time. 


Distributed  or  Stand-Alone  Simulations 

AOC  participates  in  various  distributed  simulation  frameworks.  AOC  can  participate  in 
High-Level  Architecture  (HLA)  distributed  simulation  federations  that  support  its  proprietary 
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Simulation  Object  Model  (see  Appendix  A).  AOC  is  also  designed  to  concurrently  (or 
separately)  play  in  a  non-HLA  confederation  that  includes  the  Satellite  Tool  Kit  (STK)  from 
AGI  and  FTI’s  ATR  framework. 

When  desired,  AOC  may  operate  in  a  completely  independent,  or  stand-alone  manner. 
An  option  is  provided  to  import  target  detection  data  directly  into  AOC  from  a  text  file,  and 
ATO’s  and  revised  routes  may  also  be  generated  into  text  files. 


Virtual  &  Constructive  Simulation  Modes 

AOC  is  able  to  participate  in  both  virtual  and  constructive  simulations.  Invoking  AOC  with 
a  command-line  string  launches  it  in  constructive  mode,  and  starting  AOC  from  within  the 
typical  Windows  environment  (with  a  double-click  or  Run  command)  launches  it  in  virtual 
mode. 

Flexible  Simulation  Configuration 

AOC  offers  significant  flexibility  in  how  it  behaves  during  simulation  execution,  via  its  many 
configuration  options.  Configuration  settings  may  be  saved,  to  allow  AOC  to  be  run  in  exactly 
the  same  controlled  environment  for  any  future  simulation  experiments.  Users  are  able  to 
choose  options  for: 

•  Simulation  time  (real-time,  scaled  real-time,  or  no  clock) 

•  Connections  to  other  simulation  participants  (configurable  socket  connections  for  the  STK 
simulation  confederation,  or  HLA  settings  for  federations) 

•  Behavior  of  DISE  decision-making  (factor  weights), 

•  DISE  results  usage  (always  use  DISE  selected  assets,  wait  for  human  consideration  and 
selection) 

•  Routing  (stick-routes,  pre-generated  OPUS  routes,  dynamic  OPUS  routes) 

Traecability 

AOC  offers  many  options  and  tools  for  displaying  the  progression  of  simulation  sessions. 
These  include  multiple  data  views  to  review  intel  at  various  stages  of  C2  processing,  such  as 
initial  target  lists  from  an  initial  ATO  shred,  current  updated  nominated  target  lists,  and 
nominated  target-weapon  pairings.  Additionally,  the  user  is  able  to  review  logs  of  both  the 
HLA  and  non-HLA  subsystems,  as  well  as  all  internal  events  occurring  during  a  session. 


2.2  AOC  Upgrades  for  GAVTB 

The  following  sections  describe  the  major  upgrades  to  AOC  performed  under  the  GAVTB 
contract. 


2.2.1  HLA  interface 
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AOC  Model  offers  the  capability  of  participating  in  HLA  federations.  The  HLA  interface 
(Figure  2-2)  in  AOC  Model  supports  publishing  and  subscribing  to  Aircraft  and  Target  objects, 
and  publishing  ATO  and  ATO_R  interaction  messages  which  contain  Aircraft  routing 
information  based  on  Air  Tasking  Orders  and  revisions  to  such.  AOC  Model  is  user- 
configurable  in  terms  of  how  it  drives  or  reacts  to  simulation  time  as  managed  by  HLA:  it  may 
be  configured  to  be  constrained  by  simulation  time,  it  may  be  configured  to  regulate  simulation 
time,  neither,  or  both.  While  participating  in  GAVTB  experiments  and  demonstrations,  AOC  is 
generally  configured  to  Subscribe  to  Aircraft  and  Targets,  and  to  Publish  ATO  and  ATO_R 
interaction  messages.  Additionally,  AOC  Model  was  configured  to  both  be  Constrained  and 
Regulating  with  regard  to  simulation  time  management. 


Figure  2-2  Configuration  of  AOC  Model’s  HLA  Interface 

In  addition  to  the  necessary  description  of  data  and  messages  provided  by  the  simulation 
Federation  Object  Model  (FOM),  successful  participation  in  an  HLA  federation  often  requires 
an  agreed-upon  protocol  of  events  that  define  federates’  behaviors.  AOC  Model’s  HLA 
interface  design  incorporates  such  an  agreed-upon  protocol;  it  is  referred  to  as  the  Synch 
Point  Protocol.  Synch  Point  Protocol  insures  that  AOC  Model  does  not  advance  in  simulation 
time  until  it’s  been  notified  that  all  the  other  federates  it  requires  have  joined,  and  that  it  has 
received  a  particular  synch  point  which  triggers  creation  and  dissemination  of  the  Initial  ATO. 
For  GAVTB,  the  only  required  federate  is  Suppressor.  Figures  2-3  and  2-4  illustrate  the  event 
sequences  in  AOC  and  Suppressor  federates  and  the  HLA  Run-Time  Infrastructure  (RTI) 
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which  embody  dissemination  of  the  Initial  ATO  (the  ATO  interaction),  and  revisions  to  ATO 
routes  that  occur  thereafter  (the  ATO_R  interaction). 


Figure  2-3.  Event  sequence  for  dissemination  of  Initial  ATO  by  AOC  in  GAVTB 


Figure  2-4.  Event  sequence  for  dissemination  of  revised  ATO  (ATO_R)  by  AOC 


2.2.2  Decision  Integrated  Support  Environment  (DISE) 

In  describing  the  tasks  performed  by  the  AOC,  Air  Force  regulation  AFI  13-1  AOC  states 
“The  prosecution  of  Time  Sensitive  Targets  (TSTs)  is  one  of  the  most  challenging  tasks  of  the 
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AOC’s  Combat  Operations  Division  (COD)”.  Therefore,  when  modeling  the  functions  of  the 
AOC,  we  initially  focused  on  the  operations  involved  with  the  dynamic  retasking  of  aircraft 
during  the  execution  of  the  established  ATO  based  on  new  intelligence  from  ISR  sources  ...  in 
essence  the  Time  Critical  /  Sensitive  Target  (TCT  /  TST)  challenge.  We  have  developed  and 
embedded  a  Bayesian  Network  (BN)  decision  support  tool  within  AOC.  Bayesian  Networks 
are  an  extension  of  Bayesian  decision  theory;  a  technique  used  extensively  in  business  to 
identify  the  "best"  decision  given  probabilities  of  events  or  factors. 

In  applying  BN’s  to  the  TCT  /  TST  issue,  we  focused  on  two  important  decisions.  The 
first  one  consists  of  deciding  whether  to  nominate  a  target.  If  the  target  has  been  nominated, 
the  second  part  is  deciding  the  appropriate  aircraft  to  attack  the  target. 

Once  a  target  has  been  detected,  it  is  not  automatically  nominated  for  attack.  The 
decision  of  whether  to  nominate  a  TCT  depends  on  several  factors: 

•  Classification  Confidence-represents  the  level  of  certainty  that  the  target  has  been 

correctly  identified 

•  Time  Criticality  of  TCT-how  “critical”  it  is  to  attack  the  target  depending  on  its 

operational  status 

•  TCT  Type-different  types  of  TCTs  will  have  different  priorities 

•  TCT  Target  Priority-dependent  on  the  TCT’s  time  criticality  and  its  type 

•  TCT  Mission  Priority-dependent  on  the  classification  confidence  and  the  TCT  priority 

•  Priority  of  Lowest  Assigned  Mission-represents  the  priority  of  the  least  urgent  mission 

which  has  already  been  assigned 

The  two  predominant  factors  that  influence  this  decision  are  the  priority  of  the  new 
mission  (to  attack  the  newly  spotted  target)  and  the  priority  of  the  lowest  mission  that  has 
already  been  assigned.  These  two  priorities  will  be  compared  and  the  target  will  be 
nominated  only  if  the  priority  of  the  new  mission  is  higher  than  that  of  the  lowest  assigned 
mission. 

Once  a  new  target  is  nominated,  aircraft  must  be  reassigned  to  attack  it.  Because  there 
are  many  aircraft  available,  each  aircraft  must  be  considered  and  the  most  suitable  will  be 
chosen.  This  will  depend  on  the  following: 

•  Fuel  Level-expected  amount  of  fuel  in  aircraft  at  completion  of  reassigned  mission 

•  Type  of  Weapon-available  weapons  on  board  aircraft 

•  Probability  of  Survival-based  on  the  selected  route  to  the  new  target 

•  Potential  Collateral  Damage-amount  of  damage  incurred  on  TCT  surroundings 

•  Timeline  Status  of  TCT-how  soon  it  must  be  attacked,  depending  on  its 
operational  status 

•  Time  to  Target-aircraft’s  distance  (in  minutes)  from  the  TCT 

•  Current  Mission  Priority-priority  of  the  mission  to  which  the  aircraft  is  currently 
assigned 

•  TCT  Mission  Priority-determined  when  the  target  is  originally  nominated 

•  Possible  Reassignment-represents  whether  it  is  possible  to  reassign  the  current 
target  assigned  to  this  aircraft 

•  Aircraft  Retasking  Availability-represents  any  factor  not  taken  into  account  by  the 
model,  including  commander  override 

In  this  decision,  all  of  the  factors  listed  above  will  directly  influence  the  decision. 
However,  the  relative  importance  of  each  factor  will  vary  for  different  TCTs  and  different 
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operational  situations.  For  example,  for  many  missions  Probability  of  Survival  (Ps)  may  be  the 
most  important  consideration,  but  in  a  mission  prosecuting  a  TEL  thought  to  be  preparing  to 
launch  nuclear  weapons  time  to  target  may  indeed  become  the  highest  priority.  To  take  this 
into  account,  a  weight  is  associated  with  each  factor.  Therefore  in  the  case  above,  the  weight 
specified  for  Ps  will  be  lower  in  the  case  when  a  nuclear  TEL  is  nominated. 

Within  AOC,  the  Bayesian  Decision  networks  used  to  make  the  above  decisions  are 
modeled  in  a  COTS  tool  called  Netica.  Decision  networks  can  be  represented  as  graphs  with 
nodes  and  links.  The  nodes  represent  the  different  factors  that  are  included  in  the  decision 
process;  the  links  represent  the  causal  relationship  between  the  nodes.  Three  types  of  nodes 
are  used  in  decision  networks: 

•  Nature  Nodes  -  these  nodes  are  factors  determined  by  nature,  such  as  the  fuel  level 
in  an  aircraft.  Nature  nodes  have  different  states,  which  influence  the  decision  being 
made.  Nature  nodes  are  in  the  shape  of  an  oval  or  a  white  rectangle  in  the  networks 
below. 

•  Utility  Nodes  -  the  value  (“goodness”)  of  each  combination  of  the  states  contained  in 
the  nature  and  decision  nodes.  Utility  nodes  are  seen  in  the  shape  of  diamonds. 

•  Decision  Nodes  -  used  to  recommend  /  make  the  “best”  decision.  Decisions  are 
measured  by  the  value  of  each  state  contained  in  the  node.  The  state  with  the 
highest  value  indicates  the  best  decision.  A  decision  node  is  represented  as  a  blue 
rectangle. 


2.2.3  ORCA  PLANNING  AND  UTILITY  SYSTEM  (OPUS) 

The  ORCA  Planning  and  Utility  System  (OPUS)  is  an  interactive  military  aircraft  mission 
planning  tool.  Its  autorouting  and  analysis  functions  make  OPUS  useful  for  mission 
effectiveness  and  survivability  studies.  The  system  performs  force  level  planning  as  well  as 
generating  terrain  aware  threat  avoiding  individual  sortie  routes.  OPUS  optimizes  in  the  target 
area  including  sensor  pointing  and  weapon  release  maneuvers.  OPUS  includes  utility 
functions  for  manipulating  terrain  information,  threat  data,  weapon  characteristics,  vehicle 
performance  data,  and  route  plans.  Validated  performance  and  threat  data  to  run  the  model  is 
available  from  ASC/ENS  at  Wright-Patterson  Air  Force  Base  in  Ohio. 

OPUS  is  used  for  both  operational  and  analytical  applications.  The  incorporation  of 
functionality  to  parse  Air  Tasking  Orders  (ATO)  has  made  OPUS  a  valuable  tool  for  the  USAF 
Combat  Air  Force  which  has  licensed  it  for  use  at  every  operational  Wing.  This  functionality 
(sometimes  referred  to  as  the  Hill  ATO  Defragger)  is  available  in  OPUS  version  2.47  and  later. 

Features  of  OPUS  include: 

•  An  autorouter  that  produces  threat  avoiding,  goal  seeking,  terrain  aware  routes  for 
both  conventional  and  Low  Observable  (LO)  Radar  Cross  Section  (RCS)  signatures. 

•  New  threat  analysis  techniques  result  in  route  generation  speeds  far  faster  than 
traditional  time  step  /  ray  trace  approaches. 

•  A  variety  of  figures  of  merit  for  attrition  analysis  and  a  documented  C3I  model.  A  SAM 
engagement  model  and  an  Al  endgame  model  are  implemented  for  both  Monte  Carlo 
simulation  and  static  attrition  analysis. 

•  Automatic  planning  of  weapon  releases  that  conform  to  common  tactics  for  various 
weapons  including  standoff,  gravity,  PGM,  and  interdependent  platform  /  smart 
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weapon  planning.  Eliminates  the  need  for  manual  weapon  release  sequence 
placement. 

•  Automatically  plans  weapon  release  maneuvers  including  release  heading  constraints, 
straight  and  level  times,  damage  assessment,  navigational  update,  and  offset  aim 
point  imaging. 

•  Support  for  importing  routes  from  AFMSS  and  exporting  OPUS  routes  and  threat 
laydowns  to  AFMSS. 

•  Exports  routes  in  the  Enhanced  Air  Defense  Simulation  (EADSIM)  format. 

•  Multiple  resolution  terrain  model  limits  amount  of  terrain  data  required  without 
sacrificing  resolution  in  critical  areas.  OPUS  models  terrain  avoidance  as  well  as 
defensive  and  offensive  terrain  masking  effects.  OPUS  can  develop  TF/TA  routes. 

•  Performs  automatic  search  planning  through  relocatable  target  areas. 

•  Allows  optimization  and  analysis  of  event  driven  signatures  (e.g.,  bomb  bay  doors 
opening,  onboard  jammers). 

•  The  ability  to  import  and  export  Air  Tasking  Orders  in  USMTF  AT095  and  AT098 
formats. 

FTI  has  completed  integration  of  OPUS  into  AOC.  This  was  accomplished  by  embedding 
the  OPUS  application  programming  interface  (API)  directly  into  the  AOC  code.  The  AOC 
code  was  modified  to  provide  the  data  required  by  OPUS,  and  to  utilize  the  resulting  OPUS 
output,  thereby  making  OPUS  usage  transparent  to  the  user.  OPUS  is  used  within  AOC  to 
perform  three  primary  functions: 

•  Generate  initial  aircraft  routing.  The  ATO  produced  by  AOC  contains  detailed  routing 
information  for  all  aircraft  (often  provided  at  the  wing  or  squadron  level  in  the  real 
world).  By  using  OPUS,  realistic  threat  avoiding,  terrain  aware  routes  can  be  rapidly 
generated,  taking  full  advantage  of  each  aircraft’s  unique  signature  and  weapon 
delivery  requirements. 

•  Generate  rerouting  alternatives  for  TCT  prosecution.  As  discussed  above,  when  the 
DISE  module  determines  that  a  new  target  is  critical  enough  to  modify  the  existing 
ATO,  the  next  step  is  to  evaluate  the  alternatives  for  prosecuting  that  target.  OPUS  is 
called  to  prepare  threat-avoiding  routes  to  the  new  target  for  all  aircraft  under 
consideration.  OPUS  then  provides  time  to  target  based  on  realistic  rerouting 
considerations  (not  just  stick  routes  to  the  target).  Opus  also  provides  probability  of 
survival  and  expected  fuel  status  at  mission  completion.  This  information  is  essential 
to  allow  DISE  to  select  the  best  aircraft  for  reassignment. 

•  Input  and  output  ATO’s  in  real  world  formats.  The  OPUS  ATO  parser  is  has  the  ability 
to  import  and  export  Air  Tasking  Orders  in  USMTF  AT095  and  AT098  formats.  This 
enables  ATO’s  produced  by  real  world  systems  such  as  TBMCS  to  be  used  easily 
within  the  AOC  simulation  framework. 


3.0  FTI  Support  for  GAVTB  2k  Experiment 

The  GAVTB  2k  experiment  was  designed  to  demonstrate  the  emerging  capabilities 
discussed  above,  as  well  as  validate  the  GAVTB  virtual  test  bed  reconfigurable  distributed 
simulation  concept.  As  figure  3-1  shows,  the  GAVTB  framework  was  expanded  to  include 
several  additional  components,  including  the  SST  SBR,  a  high  fidelity  space  based  radar 
model  from  Philips  Labs,  and  SIRE  /  CART  human  factors  models. 
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Figure  3-1  GAVTB  2k  Experiment  Components 

The  GAVTB  2k  experiment  included  both  a  real  time  virtual  demonstration  and 
constructive  simulation.  Figure  3-1  illustrates  the  layout  of  components  in  the  virtual 
demonstration,  using  the  HLA  RTI  as  the  backbone  for  connectivity.  The  key  to  making 
this  federation  work  was  getting  the  AOC  and  Real  Time  Suppressor  to  communicate  via 
the  RTI.  This  was  accomplished  by  a  joint  effort  from  FTI  and  SAIC.  FTI  was  then 
responsible  for  the  interfaces  from  AOC  to  STK,  OPUS,  and  SST.  SAIC  handled  the 
interfaces  to  FRED,  CART  and  SIRE.  In  the  constructive  runs,  STK,  OPUS  and  AOC 
communicated  with  Suppressor  via  a  resource  agent  developed  by  SAIC,  using  the 
Knowledge  Konnect  workflow. 


3.1  ISR  System  Simulation 

ISR  system  modeling  for  the  GAVTB  2k  experiment  was  provided  by  FTI’s 
Surveillance  Analysis  and  Simulation  Environment  (SASE)  Module.  The  core  of  the  SASE 
module  is  a  COTS  tool,  the  Satellite  Toolkit  (STK)  developed  by  Analytical  Graphics,  Inc. 
(AGI).  This  core,  and  enhancements  developed  by  FTI,  allowed  the  simulation  of  a  space- 
based  radar  constellation  and  airborne  ISR  assets,  and  to  analyze  the  performance  of 
these  ISR  assets  in  detecting  ground  based  targets,  with  focus  on  TCT  operations 
simulation.  SASE  is  designed  to  interface  with  several  AOC  components,  including  the 
aircraft  routes  generated  by  the  OPUS  module  discussed  above  and  the  DISE  module 
integrated  in  the  AOC  Module. 


3.2  SST  SBR  GMTI  Model  Integration 

The  overall  SASE  approach  to  ISR  representation  is  based  on  the  premise  of  variable 
fidelity  modeling.  Under  this  approach  the  low  to  medium  fidelity  sensor  and 
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communications  tools  included  with  STK  are  used  whenever  they  provide  an  adequate 
representation  for  the  given  analysis  task  or  experiment  underway.  When  higher  fidelity 
tools  are  required,  the  SASE  environment  provides  an  interface  to  bring  them  into  the 
distributed  simulation.  An  example  of  this  approach  was  demonstrated  in  the  GAVTB  2k 
experiment,  when  the  high  fidelity  SST  GMTI  radar  simulation  tool  was  linked  in  to  provide 
detection  data  for  a  given  satellite,  while  STK’s  radar  model  was  used  for  the  remaining 
satellites. 

SASE  modeled  the  entire  SBR  satellite  constellation  within  STK.  The  radar  defined  in 
STK  was  an  abstraction  of  the  SST  SBR  radar.  The  higher  fidelity  SST  SBR  simulation 
was  used  for  a  single  satellite,  chosen  a  priori.  SST  took  control  of  this  satellite,  and  used 
its  SBR  model  to  determine  probability  of  detection,  sensed  position  and  velocity.  This 
data  is  then  passed  back  to  STK.  STK  in  turn  passed  this  data  to  the  AOC,  along  with 
probability  of  detection  messages  from  other  satellites  in  the  constellation.  The  ATR 
algorithm  in  the  AOC  processed  all  these  detection  messages  in  a  same  manner,  making 
the  inclusion  of  SST  transparent  to  the  AOC. 

As  shown  in  Figure  3-1,  SST  and  STK  communicate  with  the  GAVTB  RTI  via  the  AOC 
module.  The  AOC  module  communicates  with  the  RTI  and  STK,  and  STK  then  shares 
data  with  SST.  The  data  required  from  STK  by  SST  is  target  position  and  velocity.  The 
AOC  module  receives  aircraft  and  target  position  and  velocity  updates  from  Suppressor 
via  the  RTI  at  a  1  hertz  rate.  The  AOC  subscribes  to  these,  and  passes  the  data  on  to 
STK  via  SASE’s  AOC-to-STK  module.  An  additional  interface  was  developed  using  STK’s 
Connect  module,  to  get  target  position  and  velocity  updates  as  well  as  satellite  position 
from  STK  to  SST,  as  well  as  to  return  probability  of  detection,  sensed  position  and  velocity 
calculated  by  SST. 


3.3  Design  of  Experiments  (DOE) 

FTI  tailored  and  applied  a  formal  DOE  process  to  design  the  experiment  used  to 
demonstrate  the  software  configuration  discussed  above.  This  process  included  the 
following  steps: 

Stepl)  Define  the  problem  (analysis  question  to  be  answered).  Prosecution  of 
Time  Sensitive/Time  Critical  Targets  (TCTs)  was  selected  as  the  problem  to  be 
analyzed,  as  this  continues  to  be  a  major  deficiency  in  the  C3ISR  process.  To  improve 
the  prosecution  of  TCTs,  the  AOC  needs  to  improve  its  process  to  make  timely  and 
accurate  decisions  related  to  the  receipt  and  interpretation  of  real-time  ISR  data. 
Additionally,  real  time  /  Near-real  time  robust  and  accurate  ISR  data  is  needed  to 
support  the  timely  prosecution  of  TCTs. 

Specifically,  the  question  to  be  addressed  was:  What  is  the  Military  Utility  /  Value  of 
SBR  in  re-tasking  ground  attack  assets  in-flight  to  prosecute  TCTs?  The  Korean 
Peninsula  was  selected  as  the  theater  of  interest. 

Step  2)  Define  the  measures  of  effectiveness  (MOEs).  Two  types  of  MOEs  were 
considered: 

•  Mission  effectiveness  measures,  including  number  of  TCTs  negated,  timeliness 
of  TCT  negation,  and  overall  value  of  target  set  engaged. 

•  Blue  force  utilization  metrics,  including  aircraft  lost  and  sorties  flown. 
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Step  3)  Identify  the  control  variables  (those  within  the  designer’s  control).  Control 
variables  considered  focused  on  blue  force  structure  variables,  including: 

•  ISR  assets  available 

o  SBR  Constellation  Performance 

■  Number  of  SBR  satellites, 

■  Deployment  Altitude, 

■  Sensor  Capabilities 

o  Number  of  UAVs,  Their  SAR  Performance,  Hi  /  Low  Altitude  Mix,  etc., 
Surveillance  Routes 

o  Other  ISR  assets  (JSTARS,  AWACS,  DSP  /  SBIRS  Hi-Lo,  Other  A/C, 
etc.) 

•  Blue  strike  aircraft  deployment: 

o  Number  of  aircraft  on  CAP,  CAP  locations 
o  Number  of  aircraft  on  ground  alert,  airbase  locations 
o  Munitions  loadout 

•  CONOPS  /  ROEs  for  nominating  targets  for  attack  and  decision  process  for 
committing  assets 

•  Timeline  Implications 

o  Comm.  Network  latency 
o  Information  Processing  latency 
o  Decision  timelines 

•  Accuracy  of  ATR  /  Target  Nomination  Process 

Step  4)  Identify  the  external  variables  (those  outside  the  designer’s  control). 
External  variables  considered  included: 

•  Theater  of  Interest:  The  Korean  Peninsula  was  selected. 

•  Threat  laydown: 

o  SAM  types  and  locations.  Although  this  experiment  was  strictly  a 
capability  demonstration,  and  not  intended  to  produce  validated  analytic 
results,  the  use  of  a  “realistic”  laydown  was  considered  important  to 
provide  “realistic”  route  re-planning  dynamics  and  implications  on  attack 
timelines. 

o  Red  Aircraft  Patrols.  For  simplicity  we  assumed  Air  Superiority  for  this 
experiment 

•  Target  locations  &  value  priorities 

•  TCT  types,  timelines,  and  movement  /  deployment  locations 

Step  5)  Design  the  layout  for  the  experiment,  and  construct  the  case  matrix.  Since 
this  was  a  capabilities  demonstration,  the  desire  was  to  keep  the  scenario  fairly  simple, 
while  still  highlighting  the  types  of  analyses  that  are  possible  with  this  GAVTB 
configuration.  The  following  entities  were  included  in  the  scenario: 

•  4  Blue  strike  aircraft,  prosecuting  an  ATO  versus  fixed  targets. 

•  0-2  Blue  strike  aircraft  on  ground  alert 

•  Blue  ISR  assets,  including  an  SBR  constellation,  and  2  global  hawk  UAVs. 

•  A  Red  bridge,  which  is  not  on  the  initial  target  list,  but  becomes  time  critical 
when  an  armored  column  is  detected  advancing  toward  it. 

•  Red  TBM  TEL,  location  initially  unknown  to  blue  forces  (pop-up  target) 

•  Red  SAM  threat  systems,  at  known  locations,  allowing  OPUS  to  plan  threat 
avoidance  routes  when  possible. 


Four  strike  aircraft  are  prosecuting  their  assigned  missions.  ISR  assets  detect 
armored  forces  northwest  of  Ich’on  approaching  a  bridge  on  the  Imjin-gang  river.  The 
bridge  is  declared  a  TCT  and  targeted  for  preemptive  strike  to  deny  crossing  of  Imjin- 
gang.  The  AOC  must  select  a  single  ship  strike  package  to  accomplish  the  mission, 
from  among  the  four  airborne  aircraft  or  the  two  aircraft  on  ground  alert.  The  choice  of 
aircraft  depends  on  time  to  target,  weapon  suitability,  sufficient  fuel,  and  the  priority  of 
the  currently  assigned  target.  ISR  assets  (SBR/UAV)  then  provide  information  on  the 
second  TST;  a  TBM  TEL  located  southwest  of  Sariwon.  AOC  again  must  re-plan  the 
flight  path  of  one  or  more  strike  aircraft  or  ground  alert  aircraft. 

Case  Matrix:  In  this  step,  we  selected  parameters  to  vary  from  among  the  control 
and  external  variables  discussed  above.  We  also  determined  the  range  over  which  to 
vary  the  selected  parameters.  Again,  the  goal  was  to  demonstrate  capability,  so  only  a 
subset  of  the  possible  control  and  external  variables  were  chosen,  and  the  range  of 
variation  was  minimal  to  keep  the  case  matrix  within  the  scope  of  the  effort.  Figure  3-2 
shows  the  selected  parameters.  The  primary  variant  was  the  SBR  constellation, 
varying  from  no  SBR,  to  a  partial  12-satellite  capability,  to  a  full  36-satellite 
constellation.  Other  variations  included  running  the  scenario  with  and  without  ground 
alert  aircraft  available,  and  changing  the  time  required  to  assimilate  process  and 
disseminate  intel  data  (the  sensor  to  decision  maker  to  shooter  timeline).  An 
additional  variable  was  the  criteria  used  to  weight  factors  used  by  the  DISE  decision 
support  module.  These  factors  include  time  to  target,  current  mission  priority, 
survivability,  and  weapon  suitability,  among  others.  Assigning  different  priorities  to 
these  factors  can  influence  aircraft  selection.  This  resulted  in  a  24  case  matrix.  The 
constructive  runs  were  performed  prior  to  virtual  demonstration. 
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Figure  3-2  Variable  ranges  for  GAVTB  2k  experiment 


Step  6)  Conduct  the  experiment,  generate  results.  As  noted  earlier,  the  constructive 
cases  were  run  using  the  Knowledge  Konnect  workflow,  and  aircraft  route  data  was 
passed  from  AOC  to  Suppressor  via  resource  agents.  FTI  components  played  the 
following  roles  in  the  demo: 

•  AOC  Model:  The  AOC  coordinated  generation  and  dissemination  of  the  initial 
ATO,  consisting  of  routes  for  four  strike  aircraft.  AOC  received  ISR  data  from 
STK,  and  performed  rudimentary  ATR  processing  to  declare  new  TCTs, 
triggering  the  DISE  module.  New  aircraft  routes  were  then  passed  to 
Suppressor.  Communication  with  Suppressor  was  done  via  Resource  Agents 
in  the  constructive  runs,  and  HLA  in  the  virtual  demo. 

•  OPUS:  OPUS  was  used  to  generate  threat  avoiding,  terrain  aware  initial 
routes  for  the  strike  aircraft,  as  well  as  potential  reroutes  for  all  aircraft  to  the 
newly  detected  TCTs.  OPUS  also  provided  survivability  and  fuel  usage 
estimates  for  the  reroutes  to  DISE. 
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•  DISE:  DISE  performed  two  functions.  As  soon  as  the  AOC  ATR  algorithms 
classified  a  new  TCT,  DISE  determined  if  it  was  high  enough  priority  to  modify 
the  existing  ATO  for  immediate  attack.  DISE  then  assessed  the  best  aircraft  to 
assign  to  the  new  TCT,  based  on  time  to  target,  survivability,  current  mission 
priority,  and  other  factors. 

•  SASE:  The  GAVTB  2k  experiment  leveraged  FTI’s  ongoing  development  of 
SASE  (Surveillance  Analysis  and  Simulation  Environment).  The  core  of  SASE 
at  this  time  is  STK,  a  COTS  software  tool.  STK  was  used  to  model  the  ground 
targets,  surveillance  aircraft,  satellites,  and  sensors.  Attack  aircraft  routes 
were  included  for  visualization.  Sensor  detections  of  TCTs  were  sent  to  the 
AOC  via  SASE  Connect  modules.  Another  important  aspect  of  SASE 
exercised  in  the  GAVTB  2k  experiment  was  the  ability  to  link  to  higher  fidelity 
external  simulations  as  required.  In  this  case,  an  interface  was  developed  to 
Philips  Lab’s  SST  SBR  GMTI  model.  SST  is  a  single  sensor  model.  The  full 
satellite  constellation  was  propagated  in  STK.  As  a  selected  satellite 
approached  the  theater,  ephemeris  data  was  transmitted  to  SST,  along  with 
target  data.  SST  then  simulated  radar  performance  for  that  satellite,  sending 
detection  data  back  to  STK.  STK  passed  this  data  along  with  detection  data 
from  the  other  satellites  to  the  AOC. 

Step  7)  Analyze  the  data,  generate  information  based  on  the  results.  All  results  from 
this  experiment  should  be  treated  as  notional.  Scenario  elements,  blue  and  red  system 
performance  parameters,  and  concepts  of  operation  employed  were  not  validated  or 
approved  by  any  government  source.  The  rigor  required  to  define  a  validated  scenario, 
as  well  as  the  resources  required  to  generate  statistically  meaningful  results  were  well 
beyond  the  scope  of  this  capabilities  demonstration.  Further,  the  experiment  was  run  at 
the  unclassified  level,  which  precluded  use  of  actual  weapon  system  performance 
parameters.  The  goal  of  the  experiment  was  to  demonstrate  the  emerging  capabilities  of 
the  GAVTB  distributed  simulation  environment,  and  the  scenario  and  data  used  were 
certainly  sufficiently  “realistic”  for  that  purpose. 

While  the  results  must  therefore  be  considered  notional,  one  of  the  capabilities  to  be 
demonstrated  was  the  robust  process  in  place  for  data  analysis  and  generation  of  military 
utility  metrics.  To  that  end,  FTI’s  l-CAIV  tool  and  process  was  used  to  archive  and 
organize  the  raw  data,  convert  results  for  each  MOE  into  a  utility  score,  and  produce  a 
notional  CAIV  (Cost  As  Independent  Variable)  plot.  The  l-CAIV  tool  for  the  GAVTB  2k 
experiment  was  structured  to  show  the  cost  -  benefit  relationship  for  the  three  SBR 
options  considered  in  performing  the  TCT  mission. 

The  l-CAIV  tool  developed  for  GAVTB  has  four  main  capabilities.  The  first  of  these  is 
the  prioritization  of  the  MOEs.  This  is  accomplished  in  the  tool  via  an  automated  AHP 
engine,  allowing  any  number  of  voters  to  enter  pair  wise  assessments  of  the  relative 
priorities  of  each  MOE.  The  l-CAIV  tool  then  uses  these  results  to  produce  a  relative 
weighting  for  each  MOE.  Furthermore,  the  tool  allows  the  user  to  assign  voters  to 
various  groups  to  see  how  the  weightings  (and  the  end  results)  vary  when  different  voter 
groups  are  considered.  Figure  3-3  shows  the  six  MOEs  measured  in  the  GAVTB 
experiment,  and  the  relative  MOE  prioritization  screen  from  l-CAIV.  As  can  be  seen, 
participants  in  the  GAVTB  MOE  prioritization  exercise  rated  preventing  TBM  launches  as 
the  highest  priority,  followed  by  minimizing  blue  aircraft  losses  and  preventing  the  armor 
column  from  crossing  the  bridge. 
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Figure  3-3  l-CAIV  MOE  prioritization 

The  second  main  capability  of  the  GAVTB  l-CAIV  tool  is  the  ability  to  convert  raw 
simulation  results  for  each  MOE  into  a  utility  score.  This  is  done  by  creating  a  tailored 
utility  function  for  each  MOE.  Figure  3-4  shows  an  example  of  a  utility  function  in  l-CAIV. 
This  example  is  for  the  MOE  representing  the  overall  value  of  the  targets  negated.  Each 
target  in  the  scenario,  including  the  TCTs,  had  a  priority  value  assigned  to  it.  This  MOE 
is  the  sum  of  the  priority  values  for  each  target  negated.  For  each  set  of  initial  conditions 
(variables  in  the  run  matrix),  l-CAIV  accesses  the  Suppressor  results  for  targets  negated, 
to  determine  the  raw  value  of  this  MOE.  Then  a  utility  function  is  used,  converting  the 
raw  score  to  a  utility  value  between  1  and  10.  The  shape  of  the  utility  function  used,  and 
the  objective  and  threshold  values  selected  can  have  a  dramatic  impact  on  the  utility 
evaluation  of  the  alternatives  under  consideration.  Therefore,  the  generation  of  utility 
functions  is  usually  done  in  concert  with  the  user  and  all  stakeholders  in  the  process. 
Two  common  types  of  utility  curves  are  shown  in  the  example;  a  linear  function  and  an  S- 
curve.  The  S-curve  provides  a  more  gradual  change  as  the  objective  or  threshold  values 
are  approached.  Other  types  of  functions,  including  logarithmic,  exponential  and  step 
functions  can  be  employed.  The  GAVTB  2k  l-CAIV  used  linear  functions  for  all  MOEs. 
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Figure  3-4  l-CAIV  utility  calculation 

After  the  raw  data  for  each  set  of  initial  conditions  for  a  given  MOE  has  been  converted 
into  utility  scores,  the  utility  scores  are  rolled  up  to  produce  utility  values  for  each 
alternative  under  consideration  for  each  MOE.  Figure  3-5  illustrates  this  for  this  same 
MOE.  At  this  point,  initial  conditions  can  be  weighted  relative  to  each  other,  so  that  more 
important  scenario  variations  have  greater  impact  on  the  rolled  up  utility  value.  For 
instance,  if  the  user  feels  that  ground  alert  aircraft  will  not  likely  be  available  for  this  type 
mission,  he  could  weight  those  cases  lower  (or  zero).  In  the  GAVTB  2k  l-CAIV,  all  initial 
conditions  were  weighted  equally. 
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Figure  3-5  l-CAIV  utility  rollup  for  a  single  MOE 

The  third  main  capability  of  the  GAVTB  l-CAIV  tool  is  the  generation  of  an  interactive 
CAIV  profile.  After  all  MOE  data  is  entered  into  the  l-CAIV  tool,  the  utility  functions 
convert  the  raw  values  to  a  common  scale,  as  described  above.  This  provides  an 
assessment  of  how  well  each  alternative  performed  for  each  MOE.  These  scores  are 
then  multiplied  by  the  appropriate  weight  for  each  MOE,  as  shown  in  Figure  3-6,  to 
produce  an  overall  utility  score  for  each  alternative. 
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Figure  3-6  l-CAIV  utility  rollup  for  all  MOEs 

Combining  this  score  with  cost  data,  the  tool  produces  the  CAIV  profile,  as  shown  in 
Figure  3-7.  The  CAIV  profile  provides  a  graphic  comparison  of  the  relative  utility  of  the 
alternatives,  as  well  as  the  cost  associated  with  each  alternative.  As  Figure  3-7  shows,  in 
the  GAVTB  2k  experiment,  the  12  satellite  constellation  produced  only  marginal  increase 
in  utility.  The  full  36  satellite  constellation  achieved  a  significantly  higher  utility,  but  at  a 
higher  cost.  Again,  the  numbers  shown  here  must  be  regarded  as  notional.  The  cost 
numbers  shown  have  no  basis  in  fact  whatsoever,  and  were  merely  added  to  illustrate  the 
functionality  of  the  l-CAIV  tool.  No  conclusions  should  be  drawn  from  this  data. 
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Figure  3-7  l-CAIV  interactive  CAIV  profile 

Note  that  this  profile  is  interactive,  in  that  the  user  can  select  any  single  MOE  (or  group 
of  MOEs),  or  MOE  prioritization  and  see  the  results  instantly  change  to  reflect  only  the 
selected  MOE(s).  Similarly,  the  user  can  select  any  subset  of  initial  conditions,  and  see 
the  impact  on  final  results.  Figure  3-8  shows  an  example  of  this,  where  the  user  has 
selected  to  see  the  utility  results  based  only  on  the  slower  timeline  latency.  These  results 
are  shown  in  orange  on  the  figure,  while  the  baseline  case  (all  conditions)  remains  in  blue 
for  comparison.  In  this  case,  it  can  be  seen  that  the  importance  of  timely  SBR  data  to  the 
AOC  is  even  greater,  since  the  longer  AOC  processing  compresses  the  time  available  to 
react  to  TCTs.  Under  these  conditions,  even  the  partial  constellation  provides  a 
significant  performance  improvement  over  no  SBR  at  all. 
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Figure  3-8  l-CAIV  interactive  CAIV  profile,  showing  sensitivity  to  initial  conditions 

The  fourth  capability  of  the  GAVTB  l-CAIV  tool  is  the  generation  of  a  three  dimensional 
CRAIV  trade  space.  CRAIV  stands  for  “Cost  and  Risk  As  Independent  Variables”.  This 
trade  space  is  created  by  importing  risk  data  for  each  concept,  to  give  the  decision  maker 
a  robust  3  dimensional  view  of  the  alternatives,  as  shown  in  Figure  3-9.  As  with  cost 
values  shown,  the  risk  values  shown  here  have  no  factual  basis,  and  were  inserted  to 
demonstrate  the  capability. 
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Figure  3-9  l-CAIV  three-dimensional  CRAIV  profile 


4.0  Conclusion 

In  conclusion,  we  feel  that  this  effort  in  support  of  the  GAVTB  effort  was  very 
successful.  Significant  new  capabilities  were  added  to  existing  components,  new 
components  were  created,  and  links  to  external  components  were  developed.  The 
GAVTB  2k  experiment  provided  a  valuable  forum  to  exercise  the  new  components  and 
links,  as  well  as  demonstrate  the  types  of  analyses  that  the  GAVTB  framework  can 
provide. 
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